April 20, 2018
I’m an Author! :O
Journey to Open Data Science, Anaconda (Continuum Analytics)
Journey to Open Data Science, Anaconda (Continuum Analytics)
cd $1 mkdir doc data src bin results cd doc echo "Doc directory with one subdirectory per manuscript" > README touch .gitkeep cd ../data echo "Data directory for storing fixed data sets" > README touch .gitkeep cd ../src echo "src for source code" > README touch .gitkeep cd ../bin echo "bin for compiled binaries or scripts" > README touch .gitkeep cd ../results echo "Results directory for tracking computational experiments peformed on data" > README touch .gitkeep echo "Folders created." cd ..
https://www.bi.vt.edu/sdal/projects/data-science-for-the-public-good-program
Blog post: From VMs to LXC Containers to Docker Containers
https://github.com/bi-sdal/dockerimages
https://github.com/bi-sdal/infrastructure_submodules
rpkgs)/rpkgs mounted in their rstudio container/rpkgs to everyone’s .libPaths()site_path = R.home(component = "home")
fname = file.path(site_path, "etc", "Rprofile.site")
write(".libPaths(c('/rpkgs', .libPaths()))", file = fname, append = TRUE) # prepend to .libPaths
write('local({r <- getOption("repos"); r["CRAN"] <- "https://cloud.r-project.org/"; options(repos=r)})',
file = fname, append = TRUE)
rpkgs: install.packages()Unless you need system libraries (e.g., CentOS)
mro Dockerfile, e.g., yum install -y jq-devel && \mro, rpkgs, rstudio, and shiny imagesdocker pushdocker push sdal/mro-c7sd_auth docker push sdal/rss-mro-c7sd_auth docker push sdal/rpkgs-mro-c7sd_auth docker push sdal/shy-mro-c7sd_auth
docker pulldocker pull sdal/mro-c7sd_auth docker pull sdal/rss-mro-c7sd_auth docker pull sdal/rpkgs-mro-c7sd_auth docker pull sdal/shy-mro-c7sd_auth
docker-compose -f rstudio-compose.yml up -d --no-recreatenginx
docker-compose.yml/rstudio-compose.yml:
volumes:
rpkgs:
services:
rstudio_chend:
image: sdal/rss-mro-c7sd_auth
container_name: rstudio_chend
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
- /etc/group:/etc/group
- /home:/home
- rpkgs:/rpkgs
- checkpoint:/checkpoint
cap_add:
- SYS_ADMIN
ports:
- 3125:8787
Virginia Tech Libraries provides free Overleaf Pro+ accounts
Write Functions! in your projects
setwd()!rstudioapi::openProject(...)here package, “A simple interface to rprojroot”: https://github.com/r-lib/here
print(R.utils::sourceDirectory(here::here('shiny', 'functions')))if (interactive()){...}rprojroot: https://github.com/r-lib/rprojrootbase::saveRDS() and base::readRDS()
base::save() and base::load()v <- 1:10 # I want to save this... save(v, file = 'awesome_datascience.RData') rm(v) load(file = 'awesome_datascience.RData') v
## [1] 1 2 3 4 5 6 7 8 9 10
saveRDS(v, file = 'super_awesome_datascience.RDS') loaded <- readRDS(file = 'super_awesome_datascience.RDS') loaded
## [1] 1 2 3 4 5 6 7 8 9 10
commit# uses console or rstudio to do password prompt
getPass::getPass("database username")
.secret_to_keep <- function(user, pass) {
if (is.null(pass)) {
pass <- getPass("LDAP Password (the one you use to login to Lightfoot and RStudio):")
}
secret_to_keep <- c(password = pass,
username = user)
return(secret_to_keep)
}
setup_user_pass <- function(username = unname(Sys.info()['user']),
password = NULL,
public_key = '~/.ssh/id_rsa.pub',
vault = '/home/sdal/projects/sdal/vault',
secret_name = unname(Sys.info()['user']),
verbose = FALSE) {
add_user(username, public_key, vault)
secret_to_keep <- .secret_to_keep(user = username, pass = password)
add_secret(secret_name, secret_to_keep, users = username, vault = vault)
}
get_my_password <- function(secret_name = unname(Sys.info()['user']),
key = local_key(),
vault = '/home/sdal/projects/sdal/vault') {
return(unname(get_secret(secret_name, key , vault)['password']))
}
I learned from last year, we should all probably just create packages for ourselves/group/lab/company
sdalr, https://github.com/bi-sdal/sdalr/blob/master/R/user_pass.R
RStatsNYC 2016: Data Testing
viscm, a tool to see how “good” your colormap is: http://bids.github.io/colormap/
Flight Rules are the hard-earned body of knowledge recorded in manuals that list, step-by-step, what to do if X occurs, and why. Essentially, they are extremely detailed, scenario-specific standard operating procedures. […]
NASA has been capturing our missteps, disasters and solutions since the early 1960s, when Mercury-era ground teams first started gathering “lessons learned” into a compendium that now lists thousands of problematic situations […] and their solutions.
— Chris Hadfield, An Astronaut’s Guide to Life.
Our planet needs our help, and we need (good) science to fix it.
— Greg Wilson
#bowtiesarecool